Video signal generating apparatus, video signal receiving apparatus, and video signal generating and receiving system

ABSTRACT

A video signal generating apparatus is disclosed which includes: a camera configured to generate a video signal; an operating device configured to operate at least either the camera or an apparatus configured to receive the video signal; a text data generating device configured to generate text data including a meaningful term in response to the operation performed on the operating device; and a text data inserting device configured to insert the text data generated by the text data generating device into the video signal in a corresponding manner.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 14/839,305, filed on Aug. 28, 2015, which claimsthe benefit of priority under 35 U.S.C. § 120 to application Ser. No.11/834,167, filed on Aug. 6, 2007, and claims the benefit of priorityunder 35 U.S.C. § 119 from Japanese Patent Application JP 2006-247179filed in the Japan Patent Office on Sep. 12, 2006, the entire contentsof each of which are hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a video signal generating apparatusequipped with a camera and to a video signal receiving apparatus forreceiving the video signal generated by that video signal generatingapparatus. More particularly, the invention relates to a system whichinserts human-readable text data into the video signal in response tooperations performed on the video signal generating apparatus or on thecamera attached thereto.

2. Description of the Related Art [Original Purpose of the EssenceMark(Registered Trademark)]

SMPTE (Society of Motion Picture and Television Engineers) stipulatescriteria called KLV (Key-Length-Value) as a basic structure in which todescribe metadata. As shown in FIG. 1, metadata is described as a bytestring of a Key-Length-Value structure. In this structure, “Key” is a16-byte label indicative of a metadata type. All keys are registered inan SMPTE metadata dictionary. “Value” denotes an actual value of themetadata indicated by the key. “Length” preceding “Value” stands for thedata length of “Value” (number of bytes).

The applicant of the present invention has proposed metadata called“EssenceMark” (registered trademark), referred to as EssenceMark™hereunder. The metadata was devised as a variation of “Term Value,” oneof the KLV metadata types according to SMPTE and used as bookmarks of AV(audio video) materials. The proposed metadata is explainedillustratively by Yoshiaki Shibata, Takumi Yoshida and MitsutoshiShinkai in “EssenceMark—SMPTE Standard-based Textual Video Marker—,”SMPTE Motion Imaging Journal Vol. 114, No. 12, pp. 463 to 473, December2005 (Non-Patent Document 1), and in “EssenceMark,” an article in theSeptember 2006 issue of the Journal of the Motion Picture and TelevisionEngineering Society of Japan Inc. (Non-Patent Document 2). The use ofEssenceMark™ involves attaching human-readable text data directly orindirectly to desired frames of an AV material. The primary objective ofEssenceMark™ is to provide high-speed access to desired locations in thematerial selectively according to text data values. Illustratively, aparticular location furnished with EssenceMark™ is instantly accessed,or all frames with specific EssenceMark™ are extracted and displayed asthumbnails (as shown in FIG. 5 of the above-cited Non-Patent Document2). The manner in which to store multi-byte characters such as those ofthe Japanese language using EssenceMark™ is precisely established.

SMPTE stipulates that for the baseband transmission of an AV material inSDI (Serial Digital Interface) format, KLV metadata packets are to bestored as ancillary data in VBI (vertical blanking interval) areas ofthe material (SMPTE 259M, 292M, 291M). As a variation of the KLVmetadata, EssenceMark™ in SDI format is thus transmitted as ancillarydata placed in the VBI areas of the frames to which the marker isattached.

Reserved words are part of the specifications of EssenceMark™. The wordswere first introduced as a common language to avert varying descriptionsof the frequently used EssenceMark™ values (i.e., values shown in FIG.1). FIG. 2 gives a list of the reserved words. Each reserved word beginswith a symbol “_” For example, the word “_RecStart” is a reserved wordthat designates a recording start point of an AV material. Wherevideotapes are patched together for consecutive recording, the recordingstart location of each videotape may be marked by this reserved word.The word “_RecEnd” is a reserved word that designates a recording endpoint of the AV material. “_ShotMark1” and “_ShotMark2” are reservedwords that specify points of interest of the AV material.

As EssenceMark™ values, any text data may be designated. Still, the mostpractical of the values are those offered as text values that have theirown meanings. FIG. 3 is the same as FIG. 2 in the above-cited Non-PatentDocument 1 (and equivalent to FIG. 1 in the above-cited Non-PatentDocument 2), schematically showing an AV material (video clips)furnished with EssenceMark™. FIG. 4 is the same as FIG. 6 in theabove-cited Non-Patent Document 1, showing a list of EssenceMark™ valuesin XML (Extensible Markup Language) attached to clip 1 in FIG. 3. Thislist of EssenceMark™ values attached to the AV material and expressed asa human-readable XML document gives an at-a-glance picture of typicalframes furnished with particular EssenceMark™ values having their ownmeanings.

Generally, when video recording equipment (e.g., optical disk recorders,VCRs) is said to comply with EssenceMark™, that means the equipment iscapable of simultaneously recording images and EssenceMark™ associatedwith the images. Earlier, this applicant proposed techniques forattaching text data such as EssenceMark™ to video materials as they arepicked up and recorded in real time so that at a later stage, relevantimage scenes furnished with the text data may be instantly accessed,displayed, or otherwise edited efficiently on an editing apparatus (seeJapanese Patent Laid-Open No. 2003-299010). However, such videorecording equipment is incapable of starting or stopping the recordingof images in accordance with EssenceMark™ attached to video materials.

[Typical Endoscope (Camera) and Video Recording Apparatus]

At present, a medical endoscope (i.e., camera) is typically connectedwith a video recording apparatus (e.g., optical disk recorder, VCR) byuse of two cables: a coaxial cable for transmitting image signals inHD-SDI format, and an RS232C cable for controlling the downstream videorecording apparatus. This setup poses no problem when the camera and thevideo recording apparatus are connected on a peer-to-peer basis.Problems can arise if the two apparatuses constitute part of a largersystem. For example, there may be a system that causes the moving imagesrecorded by the video recording apparatus to be fed to a diagnosticprogram for analysis and allows the result of the analysis to be storedinto a separate PACS (picture archiving and communication system forstorage and distribution of clinical images). In this system, the videorecording apparatus is obviously connected to a communication network.Whenever it is desired to view original moving images on a main monitorof the system, the image reproduction should be controlled by a personalcomputer carrying the diagnostic program in use; it is not practical tobring the camera up to the person who wants to view the images (it isassumed that the video recording apparatus is not at hand but controlledremotely). In other words, whereas the operations directly related toimage pickup and recording are to be controlled by the camera, unrelatedoperations such as reproduction, fast-forward, rewind, and random accessto desired locations should preferably be made through a separateinterface and not from the camera.

Typical video recording apparatuses are designed to handle all controlcommands via the RS232C cable as mentioned above. This makes itdifficult for any such apparatus to function as a subsystem in a largersystem configuration. There could be provided at least two RS232C cablesof which one may be used for the usual connection with the camera andthe other for connection with the PC carrying the diagnostic program.Such a setup, however, would demand establishing new provisionsregulating cases in which, illustratively, competing control commandsare simultaneously input through these RS232C cables (the presentprotocol has no provisions against such contingencies). The arrangementsinvolving the additional installation of redundant interface circuitswill lead to considerable cost increase. Having to install the coaxialcable and the RS232C cable between the camera and the video recordingapparatus is bothersome in the first place.

[Connection Between the Camera and the Video Recording Apparatus]

This applicant has filed another patent application (Japanese PatentApplication No. 2005-77965), undisclosed at the time of this filing. Theabove application proposes a technique to be used as follows: there arecases where a plurality of broadcasting stations are represented by asingle master station that picks up and records videos for coverage anddistributes them to the other stations coming under the master station.In such cases, the proposed technique enables the video camera of themaster station creating a material videotape to have the subordinatestations simultaneously create the same material videotape on theirVCRs. Specifically, video signals transmitted by the video camera to atleast one video recording apparatus connected to that video camera aremultiplexed with control signals. The video recording apparatus orapparatuses under the master station detect control signals from theincoming video signals and control their internal video recordingfacilities accordingly.

However, the above-cited patent application makes use of values that arenot meaningful by themselves (i.e., human-unreadable) as controlinformation. Such human-unreadable information is stored in the userarea (unstandardized and designed to accommodate user-specificspecifications) of ancillary data packets according to SMPTE 291M. Whenplaced in the user area, the human-unreadable information can only beinterpreted correctly by the video recording apparatus complying withthe proposed invention of the filed application. No other videorecording apparatus is capable of determining what the information inthe user area signifies. No interoperability exists between theseapparatuses as per different specifications. The proposed invention thushas difficulty letting its embodiment materialize beyond the scope ofapparatus-specific functional expansion.

According to the above-cited patent application, control information islimited to basic control commands (e.g., recording start, recording end,fast-forward, rewind) of the video recording apparatus. That is becausewhat is intended is solely and precisely to execute such limited tasksas enabling the master video camera creating a material videotape tohave the subordinate VCRs create the same material videotapesimultaneously.

SUMMARY OF THE INVENTION

As described, there already exist techniques that utilize EssenceMark™for instant access to scenes of interest and for listing of relevantscenes on an editing apparatus; techniques (yet to be disclosed) thatprovide video materials with human-unreadable information constitutingbasic control commands used on a video recording apparatus downstream ofthe video camera; and techniques that allow the camera such as anendoscope to control a video recording apparatus using a control cableapart from the video signal transmission cable (e.g., Japanese PatentLaid-Open No. 2004-305373).

However, there has yet to be proposed a technique that would attach tovideo materials human-readable information (i.e., meaningful by itself)in response to certain operations carried out on a camera or on anapparatus downstream thereof, so that the downstream apparatus mightutilize the attached information for performance reflecting theoperations. Nor has there been any proposed technique that would ensureinteroperability between the apparatus transmitting video materialsfurnished with such human-readable information on the one hand, andanother apparatus not complying with the technique and receiving thetransmitted video materials on the other hand.

The present invention has been made in view of the above circumstancesand provides arrangements that furnish video materials with informationmeaningful by itself in response to certain operations carried out on acamera or on an apparatus downstream thereof, so that the downstreamapparatus may utilize the furnished information for performancereflecting the operations. The invention further envisages ensuringinteroperability between an apparatus transmitting video materialsfurnished with such information according to the invention on the onehand, and another apparatus not embodying the invention but receivingthe transmitted video materials on the other hand.

In carrying out the present invention and according to one embodimentthereof, there is provided a video signal generating apparatusincluding: a camera configured to generate a video signal; an operatingdevice configured to operate at least either the camera or an apparatusconfigured to receive the video signal; a text data generating deviceconfigured to generate text data including a meaningful term in responseto the operation performed on the operating device; and a text datainserting device configured to insert the text data generated by thetext data generating device into the video signal in a correspondingmanner.

Where the video signal generating apparatus of the present invention isin use, an operation performed by the operating device on either thecamera or a downstream apparatus (i.e., video signal receivingapparatus) generates text data including a meaningful term (i.e.,human-readable information). The generated text data is inserted intothe video signal in a suitably corresponding manner.

When the text data containing a human-readable term is attached to avideo material in accordance with the operation performed on the cameraor the downstream apparatus as outlined above, the downstream apparatuscan make use of the text data to effect a process reflecting thatoperation.

According to another embodiment of the present invention, there isprovided a video signal receiving apparatus including: a receivingdevice configured to receive a video signal into which is inserted textdata including a meaningful term; a text data extracting deviceconfigured to extract the text data from the video signal received bythe receiving device; and an interpreting device configured to interpretas an operation instruction the text data extracted by the text dataextracting device; wherein the video signal receiving apparatus performsthe operation corresponding to the text data.

Where the video signal receiving apparatus of the present invention isin use, text data is first extracted from the received video signal. Theextracted text data is interpreted as an operation instruction arrangedto cause the apparatus to carry out an operation reflecting the textdata. Upon receipt of the video signal generated by the inventive videosignal generating apparatus, the video signal receiving apparatus of theinvention thus performs an operation reflecting the operation executedon the upstream apparatus.

According to a further embodiment of the present invention, there isprovided a video signal generating and receiving system made up of theabove-described video signal generating apparatus and video signalreceiving apparatus of the invention. The video signal generatingapparatus inserts text data into the video signal which is then sent toand received by the video signal receiving apparatus.

Where the inventive video signal generating and receiving system is inuse, an operation performed either on the camera of the video signalgenerating device or on the video signal receiving apparatus generatestext data including a meaningful term. The generated text data isinserted into the video signal. The video signal together with theinserted text data is received by the video signal receiving apparatus.Using the text data upon signal reception, the video signal receivingapparatus performs a process reflecting the operation carried out on thevideo signal generating apparatus.

Preferably, where the video signal generating and receiving system ofthe present invention is in use, the camera of the video signalgenerating apparatus may generate a serial digital video signal inHD-SDI format stipulated by SMPTE; the text data generating device ofthe video signal generating apparatus may generate as the text data aterm value (including EssenceMark™ as its variation) which is a type ofmetadata stipulated by SMPTE; the text data inserting device of thevideo signal generating apparatus may insert the term value as ancillarydata into a vertical blanking interval of the serial digital videosignal in the HD-SDI format; the receiving device of the video signalreceiving apparatus may receive the serial digital video signal in theHD-SDI format; and the text data extracting device of the video signalreceiving apparatus may extract the term value as the text data from anancillary data area in the vertical blanking interval of the serialdigital video signal in the HD-SDI format.

The text data containing such meaningful terms is inserted asstandardized metadata into the video signal generated by the inventivevideo signal generating apparatus. Upon receipt of the video signal thusprepared by the video signal generating apparatus, a receiving apparatusother than the video signal receiving apparatus of the invention caninterpret exactly what the included text data signifies. This ensuresinteroperability between these two apparatuses operating in tandem.

According to the video signal generating apparatus of the presentinvention, an operation performed on the camera or on the apparatusdownstream thereof generates text data containing a meaningful term thatis attached to the video material. The downstream apparatus then utilizethe text data in executing a process reflecting that operation.

The video signal receiving apparatus of the present invention isarranged to receive the video signal generated by the inventive videosignal generating apparatus. Using the text data extracted from thereceived video signal, the video signal receiving apparatus carries outa process reflecting the operation effected on the video signalgenerating apparatus.

According to the inventive video signal generating and receiving systemmade up of the video signal generating apparatus and video signalreceiving apparatus, an operation performed on the camera of the videosignal generating device or on the video signal receiving apparatusgenerates text data containing a meaningful term. Upon receipt of thevideo signal together with the text data, the video signal receivingapparatus makes use of the text data to carry out a process reflectingthe operation effected on the video signal generating apparatus.

Preferably, the term value (including EssenceMark™ as its variation)which is a type of metadata stipulated by SMPTE may be inserted into theancillary data area in the vertical blanking interval of the serialdigital video signal in the HD-SDI format. In this case, a receivingapparatus other than the inventive video signal receiving apparatus cancorrectly determine how the metadata in question is to be utilized andwhat the inserted text data signifies. This guarantees interoperabilitybetween the two apparatuses being connected.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view showing a structure of the KLV metadatastipulated by SMPTE;

FIG. 2 is a tabular view listing the reserved words of EssenceMark™;

FIG. 3 is a schematic view showing video clips furnished withEssenceMark™;

FIG. 4 is a tabular view of an XML document made up of the EssenceMark™values shown in FIG. 3;

FIG. 5 is a schematic view showing an overall configuration of a medicalvideo network system;

FIG. 6 is a block diagram showing a structure of an electronic endoscopeas part of the configuration in FIG. 5;

FIG. 7 is a flowchart of steps constituting a procedure by which the CPUin the electronic endoscope controls an SDI encoder;

FIG. 8 is a schematic view showing the EssenceMark™ value of“_RecStart”;

FIG. 9 is a block diagram showing a structure of an optical diskrecorder as part of the configuration in FIG. 5;

FIG. 10 is a flowchart of steps constituting a procedure carried out bythe CPU in the optical disk recorder in accordance with EssenceMark™values;

FIG. 11 is a schematic view of an onscreen display on a monitor as partof the configuration in FIG. 5; and

FIG. 12 is a schematic view of a variation of the onscreen display onthe monitor.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will now be described inreference to the accompanying drawings. The ensuing description willfocus primarily on how the invention is embodied illustratively in theform of an electronic endoscope for medical purposes and an optical diskrecorder for recording images picked up by the electronic endoscope.

FIG. 5 outlines a typical configuration of a medical video networksystem including the above-mentioned electronic endoscope and opticaldisk recorder. The electronic endoscope 1 and optical disk recorder 2are connected by a coaxial cable 3 that transmits a serial digital videosignal in HD-SDI format according to the SMPTE 292M standard (the signalwill be referred to as the HD-SDI signal hereunder). A video displaymonitor 4 is connected to the optical disk recorder 2.

The optical disk recorder 2 is also connected to a diagnostic PC(personal computer carrying diagnostic software) 6, an archive system(PACS: picture archiving and communication system for storage anddistribution of clinical images) 7, and an electronic clinical chartsystem 8 by way of a LAN (local area network) 5 inside the medicalfacilities.

FIG. 6 is a block diagram showing a structure of the electronicendoscope 1. The electronic endoscope 1 is made up of a scope unit(camera) 11 to be inserted into the human body and a processor unit 12,the two units being connected by an elastic tube 13. Light from a lightsource 14 inside the processor unit 12 is guided through a light guide15 inside the tube 13 and emitted from a tip of the scope unit 11. Usingthis lighting, a CCD camera (image pickup lens and CCD imaging device)16 at the tip of the scope unit 11 takes pictures inside the human body.

An image pickup signal generated photoelectrically by the CCD camera 16is forwarded to a video signal processing circuit 17 inside the scopeunit 11. The video signal processing circuit 17 subjects the imagepickup signal to digital conversion and various other signal processes(luminance signal process, color signal process, gamma correction,conversion to Y/Cb/Cr, etc.) thereby generating a digital HDTV signal.The digital HDTV signal thus generated is sent to an SDI encoder 19 inthe processor unit 12 through a parallel cable 18 inside the tube 13.

The SDI encoder 19 is a circuit that effects multiplexing, scramblingand other processes on the digital HDTV signal in accordance with theSMPTE 292M standard (HD-SDI). Under control of a CPU 20 supervising theentire processor unit 12, the SDI encoder 19 superimposes EssenceMark™as ancillary data onto a VBI (vertical blanking interval) area of thedigital HDTV signal according to the SMPTE 292M standard.

As one feature of the present invention, a control panel 21 on thehousing surface of the processor unit 12 includes a “REC” button 22, a“STOP” button 23, and a “MARK” button 24 used to operate the opticaldisk recorder 2 (FIG. 1). A microphone 25 is mounted close to thecontrol panel 21 on the housing surface of the processor unit 12.

The “REC” button 22 is a button which, when operated, causes the opticaldisk recorder 2 to start video recording. The “STOP” button 23, whenoperated, causes the optical disk recorder 2 to stop the ongoing videorecording.

The “MARK” button 24 is operated to mark a video frame of interest(e.g., a frame showing the affected region of a patient). The buttonoperation causes the optical disk recorder 2 to capture the frame inquestion as a still image.

The microphone 25 allows the operator pronouncing illustratively theword “rec,” “stop,” or “mark” to replace the physical operation of the“REC” button 22, “STOP” button 23, or “MARK” button 24, respectively. Avoice signal input to the microphone 25 is converted to an electricalsignal that is sent to a voice recognition circuit 26. The voicerecognition circuit 26 recognizes the pronounced word “rec,” “stop,” or“mark” and informs the CPU 20 of the result of the recognition.

FIG. 7 is a flowchart of steps constituting a typical procedure by whichthe CPU 20 controls the SDI encoder 19 on the basis of the operationsperformed on the “REC” button 22, “STOP” button 23, and “MARK” button 24and in accordance with the result of the recognition by the voicerecognition circuit 26.

When this procedure is started, checks are repeated in three steps untilany one of them results in the affirmative. More specifically, a checkis first made in step S1 to determine if the “REC” button 22 is operatedor if the pronounced word “rec” is recognized by the voice recognitioncircuit 26. If the result of the check in step S1 is negative, then stepS2 is reached. In step S2, a check is made to see if the “STOP” button23 is operated or if the pronounced word “stop” is recognized by thevoice recognition circuit 26. If the result of the check in step S2 isalso negative, then step S3 is reached and a check is made to see if the“MARK” button 24 is operated or if the pronounced word “mark” isrecognized by the voice recognition circuit 26. If the result of thecheck in step S3 is negative, then step S1 is reached again and theprocess is repeated.

If the result of the check in step S1 is affirmative, then step S4 isreached. In step S4, the CPU 20 generates EssenceMark™ having the value(i.e., “Value” field of the KLV metadata in FIG. 1) representative of“RecStart” (i.e., a reserved word indicating the recording start pointof an AV material as shown in FIG. 2). The CPU 20 then controls the SDIencoder 19 to superimpose the generated EssenceMark™ as ancillary dataonto the VBI area of the frame in effect at this timing. From step S4,control is returned to step S1.

If the result of the check in step S2 is affirmative, then step S5 isreached. In step S5, the CPU 20 generates EssenceMark™ having the valuerepresentative of “RecEnd” (i.e., a reserved word indicating therecording end point of the AV material as shown in FIG. 2). The CPU 20then controls the SDI encoder 19 to superimpose the generatedEssenceMark™ as ancillary data onto the VBI area of the frame in effectat this timing. From step S5, control is returned to step S1.

If the result of the check in step S3 is affirmative, then step S6 isreached. In step S6, the CPU 20 generates EssenceMark™ having the valuerepresentative of “ShotMark1” (i.e., a reserved word indicating a sceneas a point of interest in the AV material as shown in FIG. 2). The CPU20 then controls the SDI encoder 19 to superimpose the generatedEssenceMark™ as ancillary data onto the VBI area of the frame in effectat this timing. From step S6, control is returned to step S1.

According to the procedure of FIG. 7 executed by the CPU 20, the SDIencoder 19 superimposes EssenceMark™ values onto the VBI area of thedigital HDTV signal. FIG. 8 is a schematic view showing the EssenceMark™value of “RecStart” as an example of EssenceMark™ superimposed by theSDI encoder (in step S4 of FIG. 7). FIG. 8 is the same as FIG. 5 in theabove-cited Non-Patent Document 1.

As shown in FIG. 6, a parallel digital video signal processed by the SDIencoder 19 is converted to a serial signal by a parallel/serialconversion circuit 27. This serial digital video signal (i.e., HD-SDIsignal) is output from an HD-SDI output terminal 28 and sent to theoptical disk recorder 2 (FIG. 5) through the coaxial cable 3 shown inFIG. 5.

FIG. 9 is a block diagram showing that portion of the optical diskrecorder 2 which is relevant to the present invention. The HD-SDI signalsent through the coaxial cable 3 is input to an HD-SDI input terminal31. The input HD-SDI signal is converted to a parallel signal by aserial/parallel conversion circuit 32 before being forwarded to an SDIdecoder 33.

The SDI decoder 33 is a circuit that submits the converted parallelHD-SDI signal to such processes as descrambling and demultiplexing inorder to restore the original digital HDTV signal. Furthermore, the SDIdecoder 33 detects timing reference signals (EAV and SAV) and ancillarydata from the VBI area of the digital HDTV signal.

The digital HDTV signal restored by the SDI decoder 33 is forwardedthrough a superimposing circuit 34 (for superimposing characters) tothree circuits: a video signal processing circuit 35 for recordingpurposes, a still image capturing circuit 36, and a multi-screenprocessing circuit 37.

The video signal processing circuit 35 subjects the digital HDTV signalsto such processes as addition of error-correcting code, conversion to asignal format suitable for recording onto optical disks, and datacompression. The signal processed by the video signal processing circuit35 is sent to an optical pickup 38.

The still image capturing circuit 36 captures still images from thedigital HDTV signal. The data of each still picture thus captured isstored into an image memory 39 as an image file in DICOM (DigitalImaging and Communication in Medicine) format, which is a format forclinical image data.

The multi-screen processing circuit 37 generates a multi-screen videosignal that arrays, on a single screen in reduced form, the digital HDTVsignal from the superimposing circuit 34 and the still image data ofimage files (e.g., a predetermined number of the most recently createdfiles) retrieved from the image memory 39. If no image file is found inthe image memory 39, the multi-screen processing circuit 37 outputs inunmodified form the digital HDTV signal from the superimposing circuit34 (alternatively, it is possible to generate a multi-screen videosignal with its still image data portion left blank). The video signalgenerated by the multi-screen processing circuit 37 is output from amonitor display video output terminal 40 and sent to the monitor 4 inthe configuration of FIG. 5.

The timing reference signals and ancillary data detected by the SDIdecoder 33 are forwarded to a CPU 41 that controls the optical diskrecorder 2 as a whole. Based on a time code in the ancillary data, theCPU 41 controls an onscreen data generation circuit 42 to generateonscreen data (characters) indicating the current time code. Thecharacters generated by the onscreen data generation circuit 42 are sentto the superimposing circuit 34 whereby the characters are superimposedonto the digital HDTV signal.

The CPU 41 checks for the presence and type of EssenceMark™ in eachframe of the ancillary data coming from the SDI decoder 33. The CPU 41carries out the checks using the procedure shown in FIG. 10.

When this procedure is started, the checks are repeated in three stepsuntil any one of them results in the affirmative. More specifically, acheck is first made in step S11 to determine if the ancillary data of agiven frame contains EssenceMark™ having the value (i.e., “Value” fieldof the KLV metadata in FIG. 1) representative of “RecStart” (i.e., areserved word indicating the recording start point of an AV material asshown in FIG. 2). If the result of the check in step S11 is negative,then step S12 is reached. In step S12, a check is made to see if theancillary data of the frame contains EssenceMark™ having the valuerepresentative of “_RecEnd” (i.e., a reserved word indicating therecording end point of the AV material as shown in FIG. 2). If theresult of the check in step S12 is also negative, then step S13 isreached and a check is made to see if the ancillary data of the framecontains EssenceMark™ having the value representative of “_ShotMark1”(i.e., a reserved word indicating a scene as a point of interest in theAV material as shown in FIG. 2). If the result of the check in step S13is negative, then step S11 is reached again and the process is repeated.

If the result of the check in step S11 is affirmative, then step S14 isreached. In step S14, the CPU 41 controls the onscreen data generationcircuit 42 (FIG. 9) to generate suitable characters such as “REC”indicating that video recording is underway for this and subsequentframes. In step S15, the CPU 41 controls an actuator for positioning theoptical pickup 38 (FIG. 9) and a drive circuit (not shown) for driving aspindle motor to start video recording onto an optical disk (not shown)loaded in the optical disk recorder 2. This completes the processing onthe current frame.

If the result of the check in step S12 is affirmative, then step S16 isreached. In step S16, the CPU 41 controls the onscreen data generationcircuit 42 to stop generating the characters indicating that videorecording is underway (i.e., characters generated in step S14 for thepreceding frames). In step S17, the CPU 41 controls the actuator and thedrive circuit to stop the ongoing video recording onto the optical disk.This completes the processing on the current frame.

If the result of the check in step S13 is affirmative, then step S18 isreached. In step S18, the CPU 41 stores the EssenceMark™ value“_ShotMark1” into an internal memory of the CPU 41 as a document filesuch as an XML document shown in FIG. 4, along with identificationinformation (e.g., frame number) identifying the current frame. In stepS19, the CPU 41 controls the still image capturing circuit 36 (FIG. 9)to capture a still image of the current frame and to store the capturedimage as an image file into the image memory 39. This completes theprocessing on the current frame.

What follows is a description of how images are recorded to the opticaldisk recorder 2 or are displayed on the monitor 4 in response to theoperations performed on the electronic endoscope 1 in FIG. 5.

When the operator starts picking up images by operating the electronicendoscope 1, an HD-SDI signal is sent from the endoscope 1 to theoptical disk recorder 2 through the coaxial cable 3. From the opticaldisk recorder 2, the HD-SDI signal is forwarded to the monitor 4. Theimages being picked up are displayed on the monitor 4 in this manner.

To get the optical disk recorder 2 to start recording images, theoperator may either push the “REC” button 22 (FIG. 6) on the controlpanel 21 of the electronic endoscope 1 or pronounce the word “rec”toward the microphone 25 (FIG. 6) mounted close to the control panel 21.At this point, EssenceMark™ having the value of “_RecStart” issuperimposed onto the VBI area of the HD-SDI signal (in step S4 of FIG.7).

The optical disk recorder 2 separates this EssenceMark™ from the signal,interprets its value as an operation instruction, and starts recordingimages accordingly (in step S15 of FIG. 10). At the same time, suitablecharacters such as “REC” indicating that recording is underway issuperimposed onto the recorded images. The characters are also shownsuperimposed on the display of the monitor 4.

With the video recording thus started, the operator may either operatethe “MARK” button 24 on the control panel 21 (FIG. 6) or pronounce theword “mark” toward the microphone 25 upon viewing a frame of interest(e.g., a frame showing the affected region of a patient). At this point,EssenceMark™ having the value of “ShotMark1” is superimposed onto theVBI area of the HD-SDI signal (in step S6 of FIG. 7).

The optical disk recorder 2 separates this EssenceMark™ from the signal,interprets its value as an operation instruction, and stores “ShotMark1”as a document file together with the frame number (in step S18 of FIG.10). The image of the frame is captured as a still image (in step S19 ofFIG. 10). At the same time, the captured still image is shown on amulti-screen display of the monitor 4 together with the image beingpicked up.

FIG. 11 shows an onscreen display of the monitor 4 in effect when the“MARK” button 24 is pushed (or the word “mark” is pronounced) over aplurality of frames. The upper part of the screen displays the imagebeing currently picked up. On this image, the current time code and thecharacters “REC” indicating that recording is underway are shownsuperimposed. The lower part of the screen displays four still images,arrayed chronologically from left to right (i.e., the time code on theleftmost image is closest to the current time code).

Thereafter, the operator may either push the “STOP” button 23 on thecontrol panel 21 (FIG. 6) or pronounce the word “stop” toward themicrophone 25 so as to stop the ongoing video recording by the opticaldisk recorder 2. At this point, EssenceMark™ having the value of“_RecEnd” is superimposed onto the VBI area of the HD-SDI signal (instep S5 of FIG. 7).

The optical disk recorder 2 separates this EssenceMark™ from the signal,interprets its value as an operation instruction, and stops the ongoingvideo recording accordingly (in step S17 of FIG. 10). Generation of thecharacters indicating the ongoing recording is stopped (in step S16 ofFIG. 10), and the characters are no longer shown superimposed on themonitor 4.

As described, operations are carried out (physically or phonetically) onthe control panel 21 of the electronic endoscope 1 to have the opticaldisk recorder 2 start video recording, capture still images, and stopthe recording. In turn, the HD-SDI signal having EssenceMark™ as textdata containing the corresponding meaningful terms is sent to andreceived by the optical disk recorder 2. Using the EssenceMark™ textdata, the optical disk recorder 2 starts video recording, captures stillimages, and stops the recording reflecting the operations performed onthe electronic endoscope 1.

Simply connecting the electronic endoscope 1 with the optical diskrecorder 2 by a single coaxial cable 3 for HD-SDI signal transmissionmakes it possible for the endoscope to control the recorder in recordingand still image capturing.

The electronic endoscope 1 controls the optical disk recorder 2 in videorecording operation but does not control reproduction of recordedimages. The reproduction of recorded images is controlled illustrativelyby the diagnostic PC 6 (FIG. 5) via the LAN 5. Under control of thediagnostic PC 6, the optical disk recorder 2 may have its recordedimages put into files and sent out by way of the LAN 5. With thesearrangements in place, the system made up of the electronic endoscope 1and optical disk recorder 2 may be set up as a subsystem of the medicalvideo network system in FIG. 5 operating as a host system.

Since EssenceMark™ is human-readable text data as a variation of thestandardized KLV metadata, an apparatus not embodying the presentinvention such as the optical disk recorder 2 receiving the HD-SDIsignal from the electronic endoscope 1 can correctly determine how themetadata in question is to be utilized and what the text data signifies.This guarantees interoperability between the connected apparatuses (forexample, upon reproduction following the recording of images, it ispossible to display a list of thumbnail images each having theEssenceMark™ value of “_ShotMark1” superimposed thereon).

The foregoing examples have shown that the EssenceMark™ values of thereserved words listed in FIG. 2 are generated and superimposed on theHD-SDI signal. Alternatively, an EssenceMark™ value other than those ofthe reserved words may be suitably named (e.g., with disease name suchas “polyp”) and assigned to an additional control button on the controlpanel 21 of the electronic endoscope 1. Operating this control buttongenerates the EssenceMark™ value assigned to the button and causes thegenerated EssenceMark™ value to be superimposed onto the HD-SDI signal.The optical disk recorder 2, for its part, causes the onscreen datageneration circuit 42 to generate the characters corresponding to thereceived EssenceMark™ value. This allows the resulting image, with theEssenceMark™ value (“POLYP” in FIG. 12) superimposed thereon, to berecorded onto the optical disk recorder 2 and displayed on the monitor 4at the same time, as shown in FIG. 12.

As another alternative, the control panel 21 may be furnished withadditional buttons for setting the time period in which to display thesuperimposed name in characters, the locations at which to display thename, and the font color in which to display these characters. A commandindicating these settings may be added to the EssenceMark™ value in amanner distinct from the superimposed name. Given the command, theoptical disk recorder 2 may establish accordingly the time period inwhich to display the superimposed characters, the locations at which todisplay the characters, and the font color in which to display thecharacters.

In the foregoing examples, the EssenceMark™ values were shown to begenerated and superimposed by physically operated buttons or byphonetically input words. Alternatively, an EssenceMark™ value may begenerated and superimposed using one of the following three events as atrigger:

(1) a predetermined result is obtained through image processing (e.g.,the patient's affected region is detected by application of patternrecognition techniques);

(2) a predetermined result is obtained through voice processing wherebyboth image and voice are recorded (e.g., voice of a predetermined levelis detected); or

(3) a preset time is reached.

In the foregoing examples, the present invention was shown to be appliedto the electronic endoscope for medical purposes and to the optical diskrecorder for recording images picked up by the electronic endoscope.Alternatively, the present invention may be applied to varieties ofapplications covering security (crime and disaster prevention), academicpursuits (excavation of ancient ruins), and any other field where imagesgenerated by a camera are recorded by another apparatus. Furthermore,where images picked up by a camera-equipped apparatus are received byanother apparatus having no capability of video recording, thisinvention may be utilized to control the latter apparatus in itsoperation.

In the foregoing examples, the EssenceMark™ values were shown to begenerated and superimposed in response to the operations carried out onthe electronic endoscope 1 with regard to the optical disk recorder 2.Alternatively, where the invention is applied to an apparatus having acontrol unit that operates a camera, EssenceMark™ values may begenerated and superimposed in response to the operations performed onthe camera. For example, when the camera is operated to change itsfilter setting, shutter speed or gain setting, the reserved word“_FilterChange,” “_ShutterSpeedChange” or “_GainChange” as part of thoseshown in FIG. 2 may be generated and superimposed as the correspondingEssenceMark™ value.

In the foregoing examples, EssenceMark™ text data was shown generatedand superimposed onto the VBI area of the HD-SDI signal. Alternatively,it is possible to generate text data with meaningful words other thanthose of EssenceMark™, and to superimpose such text data onto videosignals in a format other than that of the HD-SDI signal.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factor in so far as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. A surgical image processing apparatus comprising:circuitry configured to generate metadata comprising recognitioninformation of an object from a surgical image data and time informationindicating a timing of the recognition of the object, associate thegenerated metadata with the surgical image data, and output the surgicalimage data with the associated metadata.